The Possibility of Combining and Implementing Deep Neural Network Compression Methods

In the paper, possibility of combining deep neural network (DNN) model compression methods to achieve better results was considered. To compare advantages and disadvantages each method, all were applied ResNet18 for pretraining NCT-CRC-HE-100K dataset while using CRC-VAL-HE-7K as validation dataset. proposed quantization, pruning, weight clustering, QAT (quantization-aware training), preserve cluster (hereinafter PCQAT), distillation performed ResNet18. The final evaluation obtained models carried out on a Raspberry Pi 4 device greatest result disk achieved by applying PCQAT whose application led reduction in size initial much 45 times, whereas acceleration via MobileNetV2 model. All model, with slight loss accuracy or an increase case clustering. INT8 quantization knowledge also significant decrease execution time.